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SECTION  I 
INTRODUCTION 


A complete  cost-effectiveness  analysis  requires  a 
combination  of  theory,  procedure  and  data.  As  a result  of  studies 
performed  on  all  three  phases,  TRG  has  planned  a series  of  reports 
on  this  subject.  These  reports  will  be  organized  in  three  groups: 
theoretical,  procedural,  and  data  acquisition.  Together  they 
comprise  a complete  analytical  method  which  will  apply  to  the  sys- 
tem and  subsystem  trade-off  analyses  to  be  performed  for  the 
Conformal /Planar Sonar  Array. 

B.  THE  THEORETICAL  PHASE 

In  the  theoretical  pha&e  of  the  program  the  following 


K 

phase 


three  areas  are  being  studied. 

(a)  The  concepts  of  system  effectiveness  from  the  view 
point  of  "mission  accomplishment". 

(b)  The  transformation  of  philosophical  concepts  into 
mathematical  expressions. 

(c)  The  mathematical  techniques  required  to  optimize 
the  cost  and  effectiveness  equations. 


The  fundamental  concepts  and  mathematical  expressions 
for  system  effectiveness  have  been  defined  by  WSEIAC  (Weapon  Sys- 
tem Effectiveness  Industry  Advisory  Committee) , and  by  NASL 
(Naval  Applied  Science  Laboratory).  They  independently  derived 
mathematical  expressions  for  system  effectiveness.  The  two 
equations  differ  in  appearance  but  are  identical  in  theory 
because  both  schools  considered  system  effectiveness  as  a func- 
tion of  availability,  dependability  and  capability  (or  performance) 
By  making  some  simplifying  assumptions  in  their  equation,  WSEIAC ^ 
was  able  to  calculate  the  effectiveness  value  of  some  simple 
weapon  systems.  The  mathematical  optimization  techniques,  however, 
were  not  discussed  in  detail. 
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Essentially,  what  is  required  now  is  some  mathematical 
techniques  with  which  one  can  manipulate  system  parameters  in 
such  a way  that  the  effectiveness  equation  is  optimized.  The 
goal  of  this  report  is  to  develop  some  general  mathematical  tech- 
niques for  optimization,  and  then  apply  these  techniques  to 
optimize  the  effectiveness  equation  of  a specific  system. 

C.  THE  PROCEDURAL  PHASE 

In  the  procedural  phase  of  the  program,  the  following 
steps  are  contemplated.  (Reports  concerned  with  all  procedural 
steps  in  system  effectiveness  analysis  will  be  submitted  to  NEL.) 

(a)  A system  block  diagram  will  be  drawn  up.  This 
block  diagram  will  show  all  major  components  (such 
as  front  end  electronics,  TR  switch,  transducers, 
beamformers,  etc.)  For  each  component,  a reasonable 
number  of  implementations  will  be  considered.  Each 
of  these  will  be  characterized  by  significant  failure 
modes.  For  each  component  considered,  at  least 
three  different  levels  of  availability  will  be  con- 
sidered, each  with  a different  cost  level. 

(b)  The  comparison  of  different  systems  will  be  made 
through  the  use  of  the  system  effectiveness  model 
developed  in  (a).  A cost  will  be  associated  with 
each  selected  configuration.  These  combinations 
will  be  examples  for  CNO , DOD  evaluation  and  con- 
venient evaluation  of  other  component  combinations 
will  be  possible. 

(c)  A reliability  program  plan  will  be  written  which 
will  contain  the  reliability  program  elements  to 
be  accomplished  by  a company  which  is  performing 
developmental  work  on  sonar  systems.  Some  examples 
of  typical  program  elements  are:  design  reviews, 
predictions,  qualification  tests,  reliability 
demonstration  tests  and  worst-case  error  analysis. 

(d)  A maintainability  program  plan  containing  the 
maintainability  program  elements  will  be  written. 
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Some  examples  of  typical  program  elements  are: 
design  review,  maintainability  demonstration  test, 
spares  test  recommendations,  plan  maintenance 
schedules. 

D.  DATA  ACQUISITION  PHASE 

The  data  acquisition  phase  will  include  the  performance 

of  the  following  tasks: 

(a)  The  analysis  of  the  predicted  values  of  reliability 
in  terms  of  constant  failure  rates  for  the  various 
environments  expected  for  that  component.  Estimated 
costs  for  each  proposed  design  alternative  will 
also  be  included.  For  those  components  where  no 
designs  yet  exist,  estimates  of  their  reliability 
will  be  generated  on  the  basis  of  the  best  avail- 
able information  using  various  standard  failure 
rate  sources. 

(b)  Repair  rates  will  be  estimated  in  terms  of  an 
assumed  constant  repair  rate.  The  repair  rates 
will  be  estimated  from  repair  time  probability 
density  functions  which  will  include  the  factors 
of  fault  detection  times,  fault  isolation  times, 
access  times,  administrative  delay  times,  removal 
times,  replacement  times,  verification  test  times, 
etc.  Instead  of  this  repair  rate  being  a function 
of  different  component  configurations,  the  repair 
rates  will  be  considered  as  a function  of  spare/ 
logistic  schemes,  number  of  available  repairmen, 
degree  of  automaticity  of  fault  isolation, 
continuous/discrete  time  testing,  number  of  test 
points,  location  of  component  on  the  ship,  degree 
of  restoration  required,  etc.  Fault  detection  and 
location  times  will  be  correlated  with,  and  taken 
from,  the  FISC  (Fault  Isolation  and  Self-Checking) 
effort  presently  pursued  by  TRG  for  NEL. 
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(c)  With  the  data  obtained  from  (a)  and  (b) , the 

availability  vector  of  the  system  will  be  found. 
Then  this  can  be  studied  as  a Markov  process,  but 
other  schemes  will  also  be  investigated.  Together 
with  the  data  obtained  from  the  reliability  and 
maintainability  program,  the  dependability  matrix 
and  the  capability  vector  will  be  formulated. 
Therefore,  the  effectiveness  equation  shall  have 
been  defined  (based  on  availability,  dependability 
and  capability)  and  optimization  will  then  be 
possible. 

E.  SUMMARY  OF  THIS  REPORT 

This  report,  being  the  first  in  the  series,  concerns 
itself  with  the  theoretical  phase  of  the  complete  analysis.  The 
goal  of  this  report,  to  repeat,  is  to  develop  some  mathematical 
techniques  with  which  one  can  manipulate  system  parameters  in 
such  a way  that  the  effectiveness  of  a system  is  optimized. 

There  are  many  techniques  to  optimize  a functional. 

Among  those  techniques  are: 

(a)  The  calculus  of  Variations. 

(b)  Theory  of  Maximum  and  Minimum. 

(c)  Dynamic  Programming  Principle. 

It  is  of  interest  to  note  that  all  three  techniques  are 
related  to  well-known  formulations  in  classical  mechanics.  The 
calculus  of  variations  is  utilized  in  the  derivation  of  the  Euler- 
Lagrange  equation,  the  theory  of  maximum  and  minimum  resulted  from 
special  consideration  of  the  Hamilton  principle,  and  the  theory  of 
dynamic  programming  evolved  from  special  considerations  of  the 
Hamilton-Jacobi  equation.  As  a result  of  these  Interesting 
relationships,  the  three  techniques  are  selected  to  illustrate 
the  nature,  and  the  solution  of  some  simple  optimization  problems. 
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SECTION  II 
DEFINITIONS 

For  the  purposes  of  a cost-effectiveness  analysis, 
a typical  system  may  be  investigated  through  the  analysis  of 
four  essential  quantitites  which  may  be  defined  as  follows: 

A.  STATE  TRANSITION  MATRIX;  R 

A state  transition  matrix  consists  of  distinguishable 
conditions  of  the  system  which  result  from  events  occurring 
prior  to  and  during  the  mission.  It  is  a function  of  re- 
liability and  repair  capability,  and  may  be  expressed  as 
a matrix: 


ii 

r12 

21 

• 

• 

• 

r22 

• 

• 

• 

nl 

rn2 

In  which  r^  (o,t)  is  defined  as  the  transition  probability 
that  the  system  is  in  state  i at  time  t,  given  state  ij  at 
time  o. 

B.  READINESS  VECTOR;  V 

The  Readiness  Vector,  V,  is  defined  as  the  probability 
that  the  system  is  ready,  and  that  at  time,  t,  all  required 
functions  are  successfully  performed.  V,  is  a function  of 
reliability,  maintenance,  and  maintenance  diagnostic  and  re- 
pair capability;  and  may  be  expressed  by: 

V — (vp  ^2*  * * ,vi'  * * ,vn^ 

In  which  v^  is  defined  as  the  probability  that  the  system  is 
in  state  i at  time  zero,  the  beginning  of  the  mission. 
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C.  MISSION  READINESS  MATRIX;  W 

The  Mission  Readiness  Matrix  W,  is  a function  of  diag- 
nosis of  system  state,  flexibility,  and  operational  policy;  and 
may  be  expressed  by: 


w^  o 

o. . 

. . . o 

O W2 

o. . 

• i «o 

o o 
• • 

w3. 

. . .o 

6 6 

o. . 

. . .w 

In  which  is  defined  as  the  probability  that  the  system  will 

be  used  for  the  mission,  given  state  i at  time  zero. 

D.  DESIGN  ADEQUACY  VECTOR;  D 

The  Design  Adequacy  Vector,  D,  is  a function  of  design 
specifications,  mission  requirements,  and  performance  capabil- 
ities; and  may  be  expressed  by: 

U 


d =1 : 


In  which  d^  is  defined  as  the  probability  that  the  ic  system 
state  will  lead  to  successful  mission  accomplishment. 


E.  SYSTEM  EFFECTIVENESS,  E 

2 

It  has  been  shown  that  the  system  effectiveness 
is  the  product  of  the  four  essential  quantities  defined  above,  or: 


n n 


E = VRWD  - £ t v.w,r.,d. 

£l  itl  1 1 J 


(1) 

2-2 


B029-41004 


Based  on  the  definitions  given  for  VRW  and  D,  it  is  apparent 
the  E is  not  only  a function  of  money,  material,  personnel 
and  time  which  are  referred  to  as  resources,  but  also  a func- 
tion of  reliability,  maintainability  and  capability  etc., 
which  are  identified  as  control  variables. 

Equation  (1),  in  its  general  form,  may  be  written  as: 
n 

E f^  (x^  * x2  * * * ®2  • • • b ) (2) 

In  which  and  t are  defined  as:  * 

A the  i—  state  of  the  system 
m^  A the  i—  control  variables 
t A time 

m^  is  a function  of  both  the  resources  and  their  derivatives. 
Mathematically : 

• • • 

m^  A x2»  * * *xn,  xl’  ^2’**°xn,  ^ C3) 


or 


x^  A h^(x^,  x2 ... , m2 . . . , t) 


(4) 


In  classical  mechanics,  the  x's  are  identified  as  the  generalized 
coordinates.  In  modem  information  theory  and  control  theory, 
the  x*s  are  referred  to  as  system  state  variables.  Equations 
(2)  and  (4)  may  be  written  in  the  following  vector  forms: 

E = F (x,  m,  t)  (5) 

X - H (x,  m,  t)  (6) 


In  which 

'el 

w 

*1 

m. 

1 

Tg 

®2 

• 

• 

i x A 

CM 

. . . . x< 

N> 

• m £ 

m r\ 
• ^ 
• 

• 

• 

en 

.XnJ 

LM 

* The  symbol  £ stands  for  "by  definition" 
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The  fundamental  concept  of  system  effectiveness  optimization  is 
to  select  a set  of  control  variables  H,  such  that  IT  is  optimized 
over  a period  of  time  defined  as  the  life  cycle  of  the  system. 

In  other  words,  the  integral: 

- rf 

E - / F(X,  M,  t)  dt  (7) 

must  be  maximized  or  minimized  with  respect  to  H. 

Equation  (7)  is  referred  to  as  the  criterion  function 
in  which  the  time  interval  tf-tQ  is  the  life  cycle  of  the  system. 
Equation  (6)  is  referred  to  as  the  dynamic  differential  equation 
of  the  system.  One  other  equation  must  be  included  in  the  anal- 
ysis. That  is,  the  constraint  equation  on  resources: 

|H|^m0  (8) 

in  which  M is  a finite  constant  vector. 
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SECTION  III 

CALCULUS  OF  VARIATIONS  APPROACH 

The  differential  calculus  deals  with  the  problem  of 
finding  points  at  which  a function  of  one  or  more  variables 
possesses  a maximum  or  minimum  value.  The  variational  calculus 
deals  with  the  problem  of  finding  paths  of  integration  for 
which  an  integral  admits  a maximum  or  minimum  value.  The  theory 
of  the  calculus  of  variations  was  formulated  by  Euler  in  the  middle 
part  of  the  18th  century.  Originally,  the  theory  was  applied 
to  solve  dynamic  problems  in  classical  mechanics.  Today,  the 
theory  finds  its  wide  range  of  applications  including  control 
theory,  electromagnetic  theory,  aerodynamics,  optics,  operational 
research  and  the  special  theory  of  relativity. 

The  following  illustrations  indicate  the  possibilities 
of  applying  the  theory  of  the  calculus  of  variation  to  deal 
with  some  problems  in  the  system  effectiveness  analysis. 

The  nature  of  the  optimization  problem  depends  on  the 
given  system  dynamic  equations 

X = H (X,  M,  t)  (9) 

If  the  boundary  condition  for  the  control  variables  are  unres- 
tricted, i.e.: 

-oo  < M < oo  (10) 


then  the  problem  of  minimizing  or  maximizing  the  integral 

- rf  - - 

E = / F (X,  M,  t)  dt  (11) 

Co 

when  the  time  limits  are  explicitly  known,  is  referred  to  as  the 
Lagrange  problem.  Two  other  fundamental  problems  in  the  calculus 
of  variations are  the  Mayer  problem  and  the  Bolza  problem.  The 
Mayer  problem  requires  us  to  find  M(t)  such  that  the  function 
G(X,M, t)  evaluated  at  the  end  points  is  minimum  or  maximum,  i.e.: 
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E * G(X,M,t) 


(12) 


- fco 


The  Bolza  problem  requires  us  to  find  M(t)  such  that  the  function: 


E = G(X,M,t) 


t'-t 


F(X,H,t)  dt 


(13) 


is  minimum  or  maximum.  The  Bolza  problem  is  the  most  general 
problem  in  the  calculus  of  variations  because  the  function  to  be 
optimized  contains  not  only  the  performance  criterion  function 
F(X,M,t)  but  also  the  constraint  function  G(E,M, t). 


In  order  to  demonstrate  the  strength  of  Euler's  principle, 
consider  the  Lagrange  problem  which  requires  us  to  optimize  the 
integral: 


E = 


(14) 


Referring  to  equation  (2),  M is  a function  of  X,  X and  t,  there- 
fore equation  (14)  can  be  written  as: 


E = 


(15) 


The  necessary  condition  for  minimizing  or  maximizing  equation 
(15)  is  that  the  Euler-Lagrange  differential  equation  must 
be  satisfied;  that  is: 

V - <Tt  ^ - 0 (16) 

n 

In  which F is  defined  as  ^ fi  • x^ 

i=l 

In  the  case  of  one- dimensional  problem,  equation  (16)  reduces  to: 


df 

3x 


d 

<Tt 


& 


In  which  f(x,  4,  t)  is  the  function  to  be  optimized. 
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Example  1:  A simple  case 

Consider  a simple  one  dimensional  problem  whose  dynamic 
equation  is  given  by: 

* = m (18) 


and  the  initial  conditions  are  given  by: 

x(o)  = X£ 
x(o)  = 0 


(19) 


The  problem  is  to  find  m(t)  such  that  the  performance  criterion 
integral: 

t, 

E 


/ * 


m^)  dt 


(20) 


will  be  minimized. 

This  is  the  Lagrange  problem  with 


2 2 2 ? 
f(x,m,t)  = yl  + in  = yl  + x 


(21) 


Application  of  equation  (17)  results  in: 


1?  - x = 0 


(22) 


The  general  solution  to  equation  (22)  is: 

x = AeC  + Be-t  (23) 

The  derivative  of  equation  (23)  is: 

* = AeC  - Be_t  (24) 

If  initial  conditions  are  applied  to  equations  (23)  and  (24), 
the  results  are: 


A = ® ' 7Xo 


(25) 
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and  equations  (23)  and  (24)  become 

x = XQ  cosh  t (26) 

* « XQ  sinh  t (27) 

hence  m(t)  = XQ  sinh  t (28) 

which  is  the  solution  to  the  problem.  In  some  complicated  problems 
(example  2,  below)  the  application  of  the  calculus  of  variations 
usually  leads  to  solving  a nonlinear  two-point  boundary -value 
problem.  In  which  case,  an  analytical  solution  becomes  extremely 
difficult  and  numerical  techniques  are  usually  employed  to  obtain 
approximate  answers. 

Example  2:  The  general  problem 

Consider  the  conditions  given  in  example  1 with  the 
dynamic  equation  given  in  the  general  form. 

£ = f^  (x,  m,  t)  (29) 


Equation  (20)  becomes 


e 


m,t)  + Xf  ^(x,m,  t 


>] 


dt 


(30) 


This  is  known  as  the  isoperimetric  problem  in  which  x is  the 
Lagrange  multiplier.  Equation  (30)  can  be  transformed  to  a 
standard  Lagrange  problem  by  letting 


h(x,m,t)  - f2(x,m,t)  + xf  (x,m,t) 


Equation  (17)  will  therefore  become: 

dh  d , dhx 

^ at  - ( 


(31) 


(32) 

In 


Equation  (32)  is  usually  a nonlinear  differential  equation, 
order  to  reduce  the  complexity  of  the  problem,  the  Lagrange 
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problem  Is  sometimes  transformed  to  the  Mayer  problem^  Generally, 
if  the  end  values  of  x are  specified,  the  Lagrange  problem 


e 


h(xp  x^ , t) 


dt 


(33) 


may  be  transformed  to  the  Mayer  problem  by  introducing  an  aux- 
iliary variable  x2  such  that 


*2^0* 


The  problem  then  becomes  one  of  minimizing 


e 


x2(tf)  dt  = x2 


(34) 


which  is  indeed  the  Mayer  problem. 

Examp le  3 . Application  of  General  Solution 

Consider  the  conditions  given  in  example  2 wherein  the 
dynamic  equation  is  given  by: 

x^  = -x^  + m (35) 

2 2 

Let  f(x,m,t)  = x^  + m , the  same  performance  criterion  *3  was 
given  in  example  1.  Hence,  the  integral  to  be  optimized  is 

tf 

e = J [(xj  + ®2)  + M-Xj  + m)] 

fco 

By  introducing  a new  coordinate  x2  such  that 

*2(cf)  = (xi  + ffl2)  + *('xi  + 

*2 (to)  = o 


dt 


m) 


(36) 


(37) 
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The  problem  now  becomes  one  of  optimizing 


■h 


x2(tf)  dt  = x2 


This  technique  of  transformation  was  employed  by  Pontryagin  in 
his  formulation  of  the  maximum  and  minimum  principle  which  will 
be  discussed  in  Section  IV. 


I 

I 

I 

I 

I 

! 

I 


B029-41004 


SECTION  IV 

THEORY  OF  MAXIMUM  AND  MINIMUM  APPROACH 


In  the  previous  discussion,  it  was  stated  that  one  of 
the  conditions  for  formulating  the  calculus  of  variations  prob- 
lem was  that  the  boundary  condition  for  the  control  variables 
be  unrestricted  (i.e.  from  - » to  +<»).  In  reality,  control 
variables  are  a function  of  resources,  and  are  therefore  always 
bounded.  Since  the  applications  of  the  calculus  of  variation  are 
limited  to  idealized  conditions  (i.e.  equation  10  must  be  sat- 
isfied), a different  approach  to  the  optimization  problem  is 
required.  The  theory  of  maximum  and  minimum  that  was  intro- 
duced by  Pontryagin  resolves  this  shortcoming.  In  essence,  the 
Pontryagin  problem  is  similar  to  the  Mayer  problem  except  that 
the  Pontryagin  formulation  allows  the  control  variables  to  be 
restricted,  i.e.: 

|H|£  Mq  (39) 

The  motivation  behind  Pontryagin' s formulation  is  relevant  in 
the  present  discussion.  Recall  the  isoperimetric  problem 
considered  in  example  2,  in  which  it  was  given  that 


x = fx(x,m,t) 


(40) 


and 


cf 

J p2(x'ra’t)  + *fi(x'm>t)J  dt 


(41) 


where  f2(x,m,t)  is  the  function  to  be  optimized.  If  equations 
(40)  and  (41)  are  written  in  their  general  vector  forms: 


X = F 0T,H,t)  = E fi 
i=l  1 


(X,E,t) 


and 


f [ £n+1or,n,t)+j;T1ft(XB,t)j 


dt 


(42) 

(43) 
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Cf 

[fn+l(*'H't>  + ^ dt  (44) 

Co 

and  the  same  procedure  is  followed  as  was  done  in  transforming 
the  Lagrange  problem  to  the  Mayer  problem,  then: 


xn+l(fcP  " fn+l^'®'fcf) 
*n+l<fco>  " ° 


(45) 


In  the  special  case  when  there  is  no  constraint,  equation  (44) 
becomes 


B - / WV  dt  - xn+l 


(46) 


Therefore,  the  problem  of  optimizing  the  integral  given  by 
equation  (44)  becomes  the  problem  of  optimizing  the  (n+1)  co- 
ordinate, xn+^(t^),  at  the  terminal  point.  For  example,  con- 
sider a system  whose  dynamic  equations  are  given  by 


x±  = ^CMLt)  i - 1,  2, 


•n 


(47) 


the  problem  is  to  find  a control  vector  M,  such  that  the  system 
changes  its  state  from  7(to)  = to  X(tf)  = at  a minimum 
time.  In  classical  mechanics,  this  is  immediately  recognized 
as  the  Brachistochrone  problem  in  which  the  following  integral 
has  to  be  minimized*: 


Vl+y*  L 


dx 


(48) 


When  Pontryagin's  principle  is  applied  to  the  problem,  it  is 
only  necessary  to  minimize  the  integral,  mln  p-  i dt.  with 


•'  min  Y 1 dt, 

H tJ 


*See  Appendix  I 
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equation  (47)  as  the  constraint.  The  procedure  requires  the 
transformation  of  the  Lagrange  problem  to  the  Mayer  problem  by 
letting 


*n+l  (t£)  * 1 and  x 


n+l(to>  = ° 


then. 


min 

H 


^f  *-f  t 

j 1 dt  - min  J xn+1(tf)  dt  = min  (xr+1)  ]f 

M t H i 


Since  xn+^  = of  time  means  optimization  with 

respect  to  the  new  coordinate  xn+^(t£>. 

In  other  problems  when  xn+^(t.)  is  not  equal  to  a constant, 
xn+^(t^)  may  be  expanded  into  a Taylor  series: 


*n+i  - fj2±i  * + »*n+i 

1 ' Xa  * « 


^Xl 


+ ^n+l 


dx<- 


a*n 


n 


(49) 


The  standard  procedure  of  transforming  the  Lagrange  problem  to 
the  Mayer  problem  will  yield: 


*n+l  " Vl<*f>  + b2X2(t£>  + bnxn(t£>  <50) 

Equation  (50)  may  be  written  in  the  closed  form 

Vhl^f*  = i^l  bixi^tf)  (51) 

Equation  (51)  is  referred  to  as  the  Pontryagin  function.  The 
minimum  time  example  discussed  above  is  the  special  case  of  eq- 
uation (51). 

Note  that  the  effectiveness  equation: 
n n 

E = £ £ viwiri1di  (Equation  1) 

j=l  i=l  1 1 1 


may  be  written  in  the  generalized  form: 
E " ^ fi^,m,t) 


(Equation  2) 
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Since  both  X and  M are  function  of  time,  equation  (2)  may  be 
further  simplified  to 

E - £ *i  (t)  (52) 

i=l 

The  effectiveness  equation  written  in  the  form  of  equation  (52) 
is  indeed  similar  to  the  Pontryagin  function.  For  this  reason, 
it  is  recommended  that  the  study  of  system  effectiveness  optim- 
ization should  follow  the  Pontryagin  procedure  which  will  be 
discussed  next. 


The  task  now  is  to  develop  a scheme  to  optimize  equation 
(52) . Observe  the  similarities  between  equation  (52)  and  the 
Hamiltonian 


H -Z  p 


i^l 


i l 


In  which  H is  the  total  energy  of  the  conservative  system.  Pon- 
tryagin has  proved**  that  maximizing  the  Hamiltonian  is  equivalent 
to  minimizing  equation  (52)  or  vice-versa.  The  formal  proof  of  the 
Pontryagin  principle  is  tedious  and  abstract,  but  intuitively 
one  may  visualize  that  minimizing  the  derivative  of  a function 
is  equivalent  to  locating  the  maximum  value  of  that  function. 


Examp le  4 : The  Pontryagin  Principle 

Consider  example  3 again  in  which  the  dynamic  equation 
was  given  by: 


= -x^  + m 


(53) 


The  performance  criterion  was  to  optimize  the  integral 


J (x^  + m^)  dt 


(54) 


It  was  shown  that  the  solution  may  be  obtained  by  using  the  La- 
grange multiplier.  The  purpose  of  this  example  is  to  illustrate 
the  Pontryagin  principle. 


mm 
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2 2 

^(tf)  = x^  + m 

*2(t0)  = 0 


Equation  (54)  then  becomes: 


f* 


x2(tf)  dt  = x2 


The  Lagrange  problem  in  equation  (54)  has  now  been  reduced  to  the 
Mayer  problem  in  equation  (56).  It  is  now  required  to  optimize 
x2(t^)  with  its  constraint,  namely  x^(t^). 

In  other  words  the  Pontryagin  function 

P = x2(tf)  + ^jx^tp  (57) 

must  be  optimized.  Instead  of  performing  the  optimization  pro- 
cess on  equation  (57)  directly,  one  may  apply  the  Pontryagin 
principle  to  formulate  the  Hamiltonian  and  then  optimize  the 
Hamiltonian  (Refer  to  equation  52): 

H =x2  + p^  (58) 

Substituting  equations  (53)  and  (55)  into  equation  (58) 
yields : 

H = x^  + m2  + P1(-x1  + m)  (59) 

Differentiating  equation  (59)  with  respect  to  m: 


S = 2m  + Pl  = 0 

from  which  the  optimum  control  variable,  m,  is  recognized  as: 

m Pi 

m = — j— 

The  problem  is  now  to  find  p^  The  derivative  of  pj^  is  re- 
lated to  H in  its  canonical  form: 


- - . - 
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*1  = " "2xl  + Pi  <6] 

Equations  (53) , (59) , (60)  and  (61)  together  with  the  Initial 
and  final  conditions  can  now  be  used  to  solve  for  without 
any  difficulty. 

It  was  mentioned  earlier  that  the  theory  of  maxi- 
mum and  minimum  is  applicable  to  problems  which  have  limit- 
ations on  the  control  variable  m.  The  next  example  illustrates 
this  point. 

Example  5:  Constrained  Control  Variable 

Consider  a simple  problem  wherein  the  dynamic 
equation  is  given  by: 

+ m (61 

The  performance  criterion  requires  optimization  of  the  inte- 
gral: 


■ 


x2dt 


With  the  following  limitation  on  m:  |m|<M0. 

Following  the  same  procedure  that  was  outlined  in  example  4: 
Let  2 

*2(tf)  = Xj. 

Formulating  the  Hamiltonian 


H = x 


p1(-x1  + m) 


In  equation  (65),  H is  a linear  function  of  m, 
therefore  the  maximum  or  minimum  value  of  H depends  on  the 
magnitude  of  m and  the  sign  of  p^.  If  H is  to  be  minimized, 


* A 


then  the  optimum  m is  a switching  function: 


m = MQsgn  (66) 

In  which  sgn  is  the  abbreviation  for  sign. 

The  problem  now  is  to  find  p^,  knowing  that  (from 
Hamilton's  canonical  form): 

* - -2xl  + Pl  (67) 

This  problem  is  best  solved  by  analog  simulation.  The  mech- 
anization of  equations  (62) , (64) , (66)  and  (67)  results  in 
the  configuration  illustrated  in  Figure  1. 

It  is  of  interest  to  examine  a physical  interpret- 
ation of  the  solution  to  the  problem  stated  in  example  5 in 
terms  of  system  effectiveness  analysis. 

Let: 

m = maintainability  whose  physical  limitation  is 
given  by  JmJ  < M0  man-hours 

= time  derivative  of  target  detection  error  and 
is  a function  of  the  error  and  maintainability. 

* = -x^  + m 


x2  ^ J xi<^t  is  integral  square  error.  The 
tQ  performance  criterion  requires  this 

quantity  to  be  minimized. 

p^  is  the  state  variable  of  the  adjoint  system,  or  the  control 
generator  which  receives  feedback  information  from  the  system 
output  and  generates  instantaneous  values  of  m.  Hence,  the 
cost  of  maintaining  the  system  under  the  stated  performance 
criterion  may  be  calculated  by  viewing  the  instantaneous  value 
of  m. 
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The  measuring  unit  shown  in  Figure  1 establishes  the  magni- 
tude of  the  error,  X2*  The  figure  also  provides  insight  into 
the  basic  philosophy  of  simulating  a one  dimensional  problem. 
The  same  theory  applies,  of  course,  to  multi-dimensional  prob- 
lems. Therefore,  among  other  applications,  the  Pontryagin 
principle  is  a powerful  tool  for  system  analysts  who  are  deal- 
ing with  optimization  problems . 

Although  the  theory  of  maximum  and  mimimum  discussed 
in  this  section  is  recognized  as  both  practical  and  useful,  it 
should  not  be  regarded  as  the  solution  to  all  optimization 
problems.  On  the  contrary,  the  application  of  Pontryagin' s 
principle  almost  inevitably  leads  to  the  solution  of  a set  of 
nonlinear  differential  equations  which  cannot  readily  be  solved 
by  analytical  means.  In  such  cas'es,  digital  computer  techni- 
ques must  be  employed  to  obtain  numerical  solutions.  Another 
drawback  is  that  the  final  state  of  the  system  must  be  known 
and  well  defined  in  order  to  permit  the  coordinate  transfor- 
mation. That  is,  the  transformation  from  the  Lagrange  prob- 
lem to  the  Mayer  problem.  In  many  practical  problems,  the 
final  state  of  the  system  is  usually  not  known.  Under  such 
conditions,  the  optimization  procedure  should  be  broken  down 
into  steps;  where  each  step  is  an  optimization  problem  by  it- 
self. This  process  is  known  as  dynamic  programming  and  is 
discussed  next. 
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SECTION  V 

DYNAMIC  PROGRAMMING  APPROACH 


Of  the  three  techniques  which  were  selected  for  dis- 
cussion in  this  paper,  perhaps  the  most  powerful  one  is  the 
principle  of  dynamic  programming  which  developed  by  R.  Bell- 
man. As  mentioned  in  the  closing  of  the  last  section,  the 
principle  of  dynamic  programming  finds  its  useful  applications 
in  dealing  with  step  by  step  decision-making  problems.  In  any 
large  system,  it  is  apparent  that  the  derivation  of  a system- 
atic procedure  for  obtaining  an  optimum  solution  to  the  prob- 
lem is  extremely  difficult.  For  example,  given  a difference 
equation: 

x(k  + 1)  = x(k)  + m(k)  (68) 

In  which  x is  the  state  variable,  m is  the  control  variable, 
and  it  is  assumed  that  x(o)  = XQ.  The  problem  is  to  find 
m(k)  such  that  y in  equation  (69)  is  minimum. 


x2(k) 


(69) 


Equation  (69)  may  be  written  as 

y = [x(o)]  2 + [x(l)J  2 + [x(3)]  2 (70) 

Substituting  equation  (68)  into  equation  (70)  results  in: 

y = [x(o)J  2 + [x(o)  + m(o)j  2 + (x(l)  + m(l)j  2 

+ [x(2)  + m(2)j  2 (71) 

Since  x(l)  = x(o)  + m(o)  I 

and  x(2)  = x(l)  + m(l)  = x(o)  + m(o)  + m(l)  J (72) 

Equation  (71)  becomes: 
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y = [x(o)]2  + [x(o)  + m (o)]2  + [x(o)  + m(o)  + m(l)]  2 

+ [x(o)  + m (o)  + m(l)  + m(2)J  2 (73) 

Equation  (73)  contains  four  independent  variables,  namely 
x(o),  m(o)  , m(l) , and  m(2) . The  problem  now  is  to  find  m(o) , 
m(l),  and  m(2)  such  that  equation  (69)  is  minimized.  Accord- 
ing to  the  theory  of  differential  calculus,  the  procedure 
would  be  as  follows : 

— — = 6x(o)  + 6m(o)  + 4m(l)  + 2m(2)  = (74) 

om(o) 

= 4x(o)  + 4m(o)  + 4m(l)  + 2m(2)  = 0 (75) 

= 2x(o)  + 2m(o)  + 2m(l)  + 2m(2)  = 0 (76) 

Equations  (74),  (75)  and  (76)  are  solved  by  simple  matrix 
algebra.  By  applying  differential  calculus  techniques,  the 
solution  of  the  problem  generally  leads  to  solving  a k x k 
matrix ► If  k equals  30  instead  of  3,  the  computation  is  so 
enormous  that  it  becomes  difficult  to  handle  even  with  digital 
computers.  Therefore,  a simplified  procedure  is  required  to 
handle  problems  of  this  nature . A new  procedure  may  be  derived 
by  applying  the  fundamental  principle  of  dynamic  programming  - 
the  Principle  of  Optimality^,  which  states: 

An  optimum  policy  has  the  property  that  whatever 
the  initial  state  and  the  initial  decision,  the 
remaining  decision  must  form  an  optimal-control 
strategy  with  respect  to  the  state  resulting  from 
the  first  decision. 

In  order  to  illustrate  the  principle  of  optimality, 
consider  the  same  example  discussed  above.  Rewrite  equations 
(68)  and  (69): 

x(k  + 1)  = x(k)  + m(k)  (77) 
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x(o)  = XQ 

and_  m(3)  = 0 


(79) 

(80) 


Equation  (79)  specifies  the  initial  state  of  the  system.  The 
subsequent  state  will  depend  on  XD.  Equation  (80)  specifies 
the  final  control  effect  is  zero.  The  problem  is  to  find  m(o) , 
m(l),  and  m(2)  such  that 


y - [x(o>]  2 + [x(l>]  2 + ft(2)]  2 + [x<3)]  2 (81) 

is  minimized.  The  procedure  of  applying  the  principle  of 
optimality  involves  the  division  of  the  problem  into  four 
steps.  Step  1:  let  yQ  = x(o)2,  find  m(o)  such  that  y^  is 

a minimum,  and  represent  the  optimum  yQ  by  y°  . Step  2: 

let  yx-  y°  + [x(l)]  2 (82) 


y^  = optimum  value  of  y^ 


similarly,  step  3:  let 


Y2  = optimum  value  of  y2 


and  finally,  step  4:  let 


y^  = optimum  value  of  y^ 

The  procedure  outlined  above  seems  to  be  quite  straight  for- 
ward. However,  in  step  1,  x(o)  is  already  a known  quantity, 
and  differentiating  y^  with  respect  to  m(o)  does  not  help  to 
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With  the  terminal  conditions  given 

x(°)  = XQ  (7, 

and  m(3)  = 0 • (8< 

Equation  (79)  specifies  the  initial  state  of  the  system.  The 
subsequent  state  will  depend  on  XQ.  Equation  (80)  specifies 
the  final  control  effect  is  zero.  The  problem  is  to  find  m(o) , 
m(l),  and  m(2)  such  that 


y = [x(o)]  2 + [x(l)]  2 + fc(2)]  2 + [x(3)] 


is  minimized.  The  procedure  of  applying  the  principle  of 
optimality  involves  the  division  of  the  problem  into  four 
steps.  Step  1:  let  yQ  = x(o)  , find  m(o)  such  that  y^  is 

a minimum,  and  represent  the  optimum  yQ  by  y°  . Step  2: 


?i“  + 


M 


y°  = optimum  value  of  y-^ 


similarly,  step  3:  let 

y2  = y?  + [x(2)] 


Y2  = optimum  value  of  y2 
and  finally,  step  4:  let 

y3  = y?  + [x<3>] 2 

y^  = optimum  value  of  y^ 

The  procedure  outlined  above  seems  to  be  quite  straight  for- 
ward. However,  in  step  1,  x(o)  is  already  a known  quantity, 
and  differentiating  y^  with  respect  to  m(o)  does  not  help  to 
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find  the  optimum  value  for  m(o) . On  the  other  hand,  m(3)  = 0 
Is  given  and  this  would  seem  to  suggest  a likely  point  at 
which  to  begin.  Therefore,  the  preceding  steps  should  be  re- 
versed and  modified  as  follows: 


Let 

y2  = [x(2)j  2 + [x(3>]  2 

(85) 

yl  = y2  + [x(1)]  2 

(86) 

yo  = yl  + [x<°>]  2 

(87) 

Note  that  y2  includes  two  terms  because  it  is  a function  of 
m(2)  only.  The  optimum  value  of  x(3)  is  obtained  by  having 
m(3)  = 0.  That  is  to  say,  the  term  is  already  optim- 

ized. 

Substituting  equation  (77)  into  equation  (85) : 

y2  - [*(2>]2  + [x(2)  + «>(2)]2 

y2  = [x(2)]2  + [x(2)]2  + 2x(2)m(2)  +[m(2)]2 
Differentiating  equation  (89)  with  respect  to  m(2)  yields: 

3^27  - 2*<2>  + 2”<2>  * 0 

and  if  we  let  the  optimum  m(2)  be  written  as  m°(2) , then 
m°(2)  = -x(2) 

and  the  optimum  y2  is  obtained  by  substituting  equation  (91) 
into  equation  (89) : 

y|  - [x(2>]  2 (92) 

Equation  (91)  indicates  that  the  necessary  control  signal  occurs 
at  k = 2.  Equation  (92)  provides  the  necessary  information  for 
the  next  optimization  step,  which  involves  the  use  of  equation 
(86).  Hence,  substituting  equation  (92)  into  equation  (86): 
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y = [x(2)]2  + [x(l)]2  (93) 

Since  x(2)  = x(l)  + m (1),  equation  (93)  becomes: 

yi  = 2[x(l)]  2 + 2x(l)m(l)  +[m(l)]  2 (94) 

Differentiating  y^  with  respect  to  m(l)  results  in: 

3STIT  ■ 2x(1>  + 2m(1)  = 0 <95) 

Let  m°(l)  = optimum  value  of  m(l) , then 

m°(l)  = -x(l)  (96) 

The  optimum  is  obtained  by  substituting  equation  (96)  into 
equation  (94) , hence 

Yl  = [*(1>]  2 (97) 

By  similar  procedures,  the  optimum  m(o)  and  yo  are  found  to  be: 

m°(o)  = -x(o)  (98) 

y°  - [«(«.)]  2 (99) 

The  recurrence  process  may  be  carried  out  by  a systematic 


computer  program  without  any  difficulty.  The  ease  of  digital 
solution  is  the  main  advantage  of  the  dynamic  programming 
technique  - especially  when  dealing  with  large  multi-stage 
system  optimization  problems. 

In  order  to  generalize  the  results  obtained  from  the 
preceding  example,  consider  a discrete  system  whose  dynamic 
equation  is  given  by: 

X(k+1)  = A (k)  X (k)  + M(k)  (100) 

In  which  X is  the  state  vector,  M is  the  control  vector  and  A 
is  a matrix.  The  optimization  criterion  is  given  by: 
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y = ^ F[x<k>’  R<k>] 


(101) 


Assume  the  initial  state  and  the  final  control  variables  are 
specified  as: 


X(o)  = Xr 


(102) 


M(N)  = Hn 

Then  the  optimization  of  y over  N steps  may  be  written  as 


(103) 


y°  = Min  Y F [X<k>  > fi(k>] 

m(o),  m(l),  m(2) 


(104) 


Equation  (104)  may  be  divided  into  two  steps;  with  a view  toward 
developing  a recursion  formula.  .. 


Ym  = Min 


f[x(o),M(o)]  + Y F Jx(k)  , fl(k)] 


m(o) , m(l) . . . .m(N) 


Rewrite  equation  (105) : 


y°  = Min  < f[X(o),  fi(°)]  + Min 

m(o)  m(l)  > m(2)  . . . ,m(N) 


(105) 


IN 

Z!f[x(r),  R(k 

k=l  L 


Let  7^  = Min  mm  mrn^Ft*(k)’  ^ 

m(l)  , m(2)  . . . (107) 
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Hence  equation  (106)  becomes 

y 


n ■ (F  (*<°>  • + ^S-i} 


a(o)  ^ 

Similarly  y^_^  may  be  divided  into  two  steps 


(108) 


o 


l-i  - $?){F[X<1>*  H<1>]+  ^-2} 


(109) 


In  which  y^_2  is  defined  as 


yN_2  = Min  # 

N * m(2),  m(3)....m(N)j^ 


£ 


F[X(k),  R(k)] 


(110) 


The  procedure  for  obtaining  y^.3  is  the  same  as  above. 

Hence  a recurrence  formula  based  on  equations  (104)  through 
(110)  is  developed: 

y°  = Min  <f[*(o),  FT(o)1  + Min  <f[)T(1)  , M(l)l 
m(o)  [ J m(l) [ J 

+ S(5)  {*  yN-u}} 


(111) 


It  should  be  apparent  now  that  the  preceding  example  is  a 

special  case  of  equation  (111)  in  which  F[x(k) , M(k)J  « [x(k)]^. 

Of  course,  the  dynamic  programming  technique  has  the  capability 
to  handle  much  more  complex  functions  such  as  the  quadratic 
form 

F [X(k) , H(k)]  = [X(k)  - Xd]TQ[7(k)  - Xd]  (112) 

In  which  T^is  the  desired  state,  Q is  a square  matrix  and 
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[X(k)  - **]T  is  the  transpose  of  [x(k)  - X**]. 

When  the  principle  of  dynamic  programming  applies  to  continu- 
ous systems,  equations  (100)  and  (101)  may  be  written  as 

X0C,  M,  t)  - AX  + M 


(113) 


?f 

J F(X,H, 


t)dt 


(114) 


In  parallel  with  the  procedure  used  in  discrete  systems,  sep- 
arate equation  (114)  into  two  steps: 


t +At 
,o 


J F (X,  R,  t)dt  + J F(X,  H,  t)  dt 


t0+At 


(115) 


Define  y°  * optimum  value  of  y,  such  that 

t +At 


y°[*(to>.  *o]  “ *«. 


M(t)€ 


c 


F(X,  R,  t)  dt 


u 


F (X,M,  t)dt 


(116) 


The  notation  M(t)  € (t „,tf)  means  M(t)  belongs  to  the  set 

fo  < t < t£} 

Then : ' 


y°fjr(t_  + At)»  + Atl  = Min 
L ° ° J M(t)C(t, 


rF(3T,  R, 

t0+At 


t)dt 

(117) 


and  equation  (116)  may  be  written  as: 
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'[*<««>  • Co]  - K 


*o+At 


Min 

M(t)  (t 


F(X,  M,  t)dt 


f£  1 

+ gin  F (X,  R,  t)  dt> 

H(t)  (t  + At,  tf)J  f 

Co  + At  j 


(118) 


Substitute  equation  (117)  into  equation  (118),  and 
y°[X(tD),  tQj  becomes  analogous  to  equation  (108). 

to+At 

^o).g-Mi„e(to>to  + at)  j FaB>t)dt 

to 

+ y0[x(t  + At)  t + Atl \ 

i ° Jj  (i«: 

Through  the  use  of  a Taylor  expansion  and  approximation,  equation 

(119)  can  be  reduced  to  the  Hamilton-Jacobi  equation:* 


dtrt  M(t 
o ' o 


}|f(X,  m. 


t)  + <vx  y°,  x> 


(120) 


The  symbol  < > stands  for  vector  multiplication  which  is  well 
known  in  classical  mechanics.  Current  literature  refers  to  equation 

(120)  as  the  equation  of  Dynamic  Programming.  The  method  of  solv- 
ing equation  (120)  is  best  illustrated  by  doing  an  example. 

Consider  a continuous  system,  whose  dynamic  equation  is  given  by: 


x = -ax  + m 


(121) 


See  Appendix  II 
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The  performance  criterion  is  given  by 


T 


(**  + 


m2)dt 


The  problem  is  to  find  m(t)  such  that  equation  (122)  is  mini 
mized.  From  equation  (122),  the  function  F(K,  H,  t)  is 
identified  as: 


F(X,  H,  t)  = \ (x2  + m2) 

n T 

While,  the  term  <V^  y , X>  in  equation  (120)  is  identified  as 

y°,  X>  = (-ax  + m) 

Substituting  equations  (123)  and  (124)  into  equation  (120) 
will  result  in 

- |*  = Min|j  (x2  + m2)  + (-ax  + m)j 

To  find  the  optimum  m(t),  differentiate  equation  (125)  with 
respect  to  m(t) 

(”tt)=m  + ^=  0 

Hence  the  optimum  m is  found  to  be 


Substituting  equation  (127)  into  equation  (125) : 


(122) 

(123) 

(124) 

(125) 

(126) 

(127) 

(128) 
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Equation  (128)  is  a non-linear  second  order  partial  differ- 
ential equation.  It  is  best  solved  by  the  method  of  separation 
of  variables.  Therefore  assume  a function  y°  which  contains 
the  non-linear  x term  multiplied  by  another  function  of  time: 


y°  = 7 P(t)  x2 

Upon  substituting  equation  (129)  into  equation  (128) , it 
becomes : 


(129) 


1 dp  2/1  212?  ) 

“ 7 cTt  x = \7  x +7P  (c)x  + p(t)x(-ax  - p(t)x)> 


(130) 


equation  (130)  may  be  simplified  to; 


= +p2  + 2ap  -1 


(131) 


Equation  (131)  may  be  integrated  to  give  the  following  result8: 


t 2 tanh"1  --2P  + 2a 

^a2  + 4 

After  simplification,  equation  (132)  may  be  written 

p(t)  = Va2  + 1 tanh(-t*Va2  + 1 ) -a 
Substitute  equation  (133)  into  (129)  to  get: 


(132) 


(133) 


y°  = 7 x2 


{^TT  ta„h(-tV77T)  -a\ 


(134) 


According  to  equation  (127)  the  optimum  m is  the  negative 
partial  derivative  of  y°  with  respect  to  x.  Hence 

m°  = -x  j^2  + i tanh(-t*^a^  + lj  -a^ 


(135) 
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This  example  finds  its  direct  applications  in  the  study  of 
system  effectiveness  by  considering  x as  an  error  and  m as  the 
maintainability  which  is  responsible  for  that  error.  The  op- 
timum value  for  m was  found  in  equation  (135) . The  method 
of  obtaining  m from  the  Jacobi -Hamilton  equation  may  be  sum- 
marized as  follows : 


Step  1. 
Step  2. 


Step  3 . 


S tep  4 . 


Identify  F()f,  H,t)  and  <V^  y°,  X>  for  the  given 
problem,  and  formulate  the  Hamilton-Jacobi  equation. 

Differentiate  the  Hamilton-Jacobi  equation  with 
respect  to  M,  set  the  derivative  equal  to  zero  and 
find  5?  in  terms  of  y°. 

Substitute  R as  a function  of  Vg  y°  into  the  Hamil- 
ton-Jacobi equation.  Assume  a solution  for  y°  is 
composed  of  two  functions,  time  and  x.  Solve  the 
equation  by  the  method  of  the  separation  of  variables. 

Obtain  M°  by  forming  v*y°. 


The  four  steps  outlined  above  will  permit  utilization  of  a 
digital  computer  to  provide  approximate  answers  when  analytical 
solutions  of  problems  discussed  herein  are  impractical. 
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SECTION  VI 

SUMMARY  AND  CONCLUSION 

The  three  mathematical  techniques  discussed  are: 

1.  The  Calculus  of  Variations 

2.  The  Theory  of  Maximum  and  Minimum 

3 . The  Principle  of  Dynamic  Programming 


The  Calculus  of  Variations  require  unbounded  control  variables . 
This  basic  requirement  limits  the  applications  of  the  theory  in 
the  study  of  effectiveness.  The  theory  of  Maximum  and  Minimum 
does  not  require  unbounded  control  variables,  but  it  does  re- 
quire well  defined  end-points.  This  requirement  also  limits 
the  applications  considerably.  The  Principle  of  Dynamic  Pro- 
gramming is  not  limited  by  these  requirements,  and  would  seem 
to  provide  the  most  promising  technique  to  solve  optimization 
problems  arising  from  cost-effectiveness  studies.  The  ultimate 
goal  of  the  current  investigation  will  be  the  application  of 
these  techniques  to  the  optimization  of  the  C/P  design  and  trade 
off  analysis.  The  work  will  progress  in  a general  fashion  so  as 
to  allow  either  major  sub-system  trade-offs  (e.g.,  comparison  of 
beam-forming  technique)  or  to  allow  component  trade-offs  within 
sub-systems. 


- 1^, 


* Conformal /Planar  Array  Sonar 


i 
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APPENDIX  I 


THE  BRACHISTOCHRONE  PROBLEM 


Consider  an  object  of  mass  M as  being  released  from 
rest  with  initial  velocity  equal  to  zero  as  shown  below: 


The  problem  is  to  find  the  path  from  A to  B along  which  the 
object  may  travel  in  the  least  amount  of  time. 

1 2 

The  kinetic  energy  of  the  system  is  Mv  and  the 

potential  of  the  system  is  Mgy.  In  which  v stands  for  the 
velocity  of  the  object  and  g is  the  gravitational  constant. 
Assuming  that  the  total  energy  of  the  system  in  conserved, 


Simplify  equation  (1)  and  let  v 
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APPENDIX  II 

DERIVATION  OF  T* E HAMILTON- JACOBI  EQUATION 


Rewrite  equation  (119)  : 


' t0+At 

y°[x(t  ) , C ] • Mtn  //  F(*,  H, 

° 0J  H(t)f<t  , tD  + it)  J 

> Co 


t)  dt 


+ y°[*(t0  + at).  t0  + it] 


So+At 


Note  that  the  Integral  I F(X,  H t)dt  may  be  expanded  into 


the  Taylor  series: 


tc+At 


F(X,  H,t)dt  = 


Z(tQ  + At) 


Z(t  ) + # At  + 0(At2) 
o 


Since  Z(tQ)  = 0 and  « F(X,  H,  tQ) , then  equation  (2)  becomes 

o 


t +At 
rO 


F(X,  H,  t)dt  = F(X,  H,  t0)At  + 0(AO 
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\%i 

|;:j 

r 


The  next  term  in  the  recursion  y0[i(tQ  + At) , tQ  + At]  may  also 
be  expanded  into  the  Taylor  series: 


y°[X(t0  + it),  eo  + it]  = y°[X(t0),  tj  + it[|^]. 


+ At 


Kl- 


*9  ^ At 


* 


+ At  + 0(At2) 


Equation  (4)  may  be  written  as 


n 


(4) 


y [X(to  + At),  tQ  + At]  = y°[X(to),  tj  + AtJ^|j- 

k=i  1 


dvu  2 

+ At  + O(At^) 


(5) 


f - 


* 

i* 


i 


Define 


i=l  1 


Then  Equation  (5)  becomes 


y°[3T(to  + At),  tQ  + At]  = y°[X(tQ),  tj 


At|(Vxy°)  *(*)  f + At  + 0(At2) 


(7) 


' Vi.'  • : 


wrnmmmmmmmmmm 


t 


:* 

'i 


i 


i' 
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Substitute  equations  (3)  and  (7)  Into  equation  (1) : 


y0[u(t0).  tj  - Jta  + JfS.  5.  t0)At  + y‘[*(t0>,  *J 


+ At 


t + 0(At' 


(8) 


Define  (V7  y°)  (x)  = < 7s  y°,  I > and  drop  terms  second  order  in 


At. 


Then  equation  (8)  may  be  written  as 


- At  = F(X,  M,  t0)At  + < Vy  y°,  * > At 


(9) 


Divide  equation  (9)  by  At: 


- |j£  - F(X,  M,  tQ)  + < Vjj  y°,  I > 


(10) 


Equation  (10)  is  identified  as  the  Hamilton-Jacobi  equation. 


